Differences in Discounting Behavior and Brain Responses for Food and Money Reward.

Most neuroeconomic research seeks to understand how value influences decision-making. The influence of reward type is less well understood. We used functional magnetic resonance imaging (fMRI) to investigate delay discounting of primary (i.e., food) and secondary rewards (i.e., money) in 28 healthy, normal-weighted participants (mean age = 26.77; 18 females). To decipher differences in discounting behavior between reward types, we compared how well-different option-based statistical models (exponential, hyperbolic discounting) and attribute-wise heuristic choice models (intertemporal choice heuristic, dual reasoning and implicit framework theory, trade-off model) captured the reward-specific discounting behavior. Contrary to our hypothesis of different strategies for different rewards, we observed comparable discounting behavior for money and food (i.e., exponential discounting). Higher k values for food discounting suggest that individuals decide more impulsive if confronted with food. The fMRI revealed that money discounting was associated with enhanced activity in the right dorsolateral prefrontal cortex, involved in executive control; the right dorsal striatum, associated with reward processing; and the left hippocampus, involved in memory encoding/retrieval. Food discounting, instead, was associated with higher activity in the left temporoparietal junction suggesting social reinforcement of food decisions. Although our findings do not confirm our hypothesis of different discounting strategies for different reward types, they are in line with the notion that reward types have a significant influence on impulsivity with primary rewards leading to more impulsive choices.

Attribute latencies causally shape intertemporal decisions.

Intertemporal choices - decisions that play out over time - pervade our life. Thus, how people make intertemporal choices is a fundamental question. Here, we investigate the role of attribute latency (the time between when people start to process different attributes) in shaping intertemporal preferences using five experiments with choices between smaller-sooner and larger-later rewards. In the first experiment, we identify attribute latencies using mouse-trajectories and find that they predict individual differences in choices, response times, and changes across time constraints. In the other four experiments we test the causal link from attribute latencies to choice, staggering the display of the attributes. This changes attribute latencies and intertemporal preferences. Displaying the amount information first makes people more patient, while displaying time information first does the opposite. These findings highlight the importance of intra-choice dynamics in shaping intertemporal choices and suggest that manipulating attribute latency may be a useful technique for nudging.

Choice impulsivity after repeated social stress is associated with increased perineuronal nets in the medial prefrontal cortex.

Repeated stress can predispose to substance abuse. However, behavioral and neurobiological adaptations that link stress to substance abuse remain unclear. This study investigates whether intermittent social defeat (ISD), a stress protocol that promotes drug-seeking behavior, alters intertemporal decision-making and cortical inhibitory function in the medial prefrontal cortex (mPFC). Male long evans rats were trained in a delay discounting task (DDT) where rats make a choice between a fast (1 s) small reward (1 sugar pellet) and a large reward (3 sugar pellets) that comes with a time delay (10 s or 20 s). A decreased preference for delayed rewards was used as an index of choice impulsivity. Rats were exposed to ISD and tested in the DDT 24 h after each stress episode, and one- and two-weeks after the last stress episode. Immunohistochemistry was performed in rat's brains to evaluate perineuronal nets (PNNs) and parvalbumin GABA interneurons (PV) labeling as markers of inhibitory function in mPFC. ISD significantly decreased the preference for delayed large rewards in low impulsive, but not high impulsive, animals. ISD also increased the density of PNNs in the mPFC. These results suggest that increased choice impulsivity and cortical inhibition predispose animals to seek out rewards after stress.

How do animals weigh conflicting information about reward sources over time? Comparing dynamic averaging models.

Optimal foraging theory suggests that animals make decisions which maximize their food intake per unit time when foraging, but the mechanisms animals use to track the value of behavioral alternatives and choose between them remain unclear. Several models for how animals integrate past experience have been suggested. However, these models make differential predictions for the occurrence of spontaneous recovery of choice: a behavioral phenomenon in which a hiatus from the experimental environment results in animals reverting to a behavioral allocation consistent with a reward distribution from the more distant past, rather than one consistent with their most recently experienced distribution. To explore this phenomenon and compare these models, three free-operant experiments with rats were conducted using a serial reversal design. In Phase 1, two responses (A and B) were baited with pellets on concurrent variable interval schedules, favoring option A. In Phase 2, lever baiting was reversed to favor option B. Rats then entered a delay period, where they were maintained at weight in their home cages and no experimental sessions took place. Following this delay, preference was assessed using initial responding in test sessions where levers were presented, but not baited. Models were compared in performance, including an exponentially weighted moving average, the Temporal Weighting Rule, and variants of these models. While the data provided strong evidence of spontaneous recovery of choice, the form and extent of recovery was inconsistent with the models under investigation. Potential interpretations are discussed in relation to both the decision rule and valuation functions employed.

Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex.

Deciding on a course of action requires both an accurate estimation of option values and the right amount of effort invested in deliberation to reach sufficient confidence in the final choice. In a previous study, we have provided evidence, across a series of judgment and choice tasks, for a dissociation between the ventromedial prefrontal cortex (vmPFC), which would represent option values, and the dorsomedial prefrontal cortex (dmPFC), which would represent the duration of deliberation. Here, we first replicate this dissociation and extend it to the case of an instrumental learning task, in which 24 human volunteers (13 women) choose between options associated with probabilistic gains and losses. According to fMRI data recorded during decision-making, vmPFC activity reflects the sum of option values generated by a reinforcement learning model and dmPFC activity the deliberation time. To further generalize the role of the dmPFC in mobilizing effort, we then analyze fMRI data recorded in the same participants while they prepare to perform motor and cognitive tasks (squeezing a handgrip or making numerical comparisons) to maximize gains or minimize losses. In both cases, dmPFC activity is associated with the output of an effort regulation model, and not with response time. Taken together, these results strengthen a general theory of behavioral control that implicates the vmPFC in the estimation of option values and the dmPFC in the energization of relevant motor and cognitive processes.

Dopamine activity in the nigrostriatal pathway alters cue-induced risky choice patterns in female rats.

Deficits in cost/benefit decision making is a critical risk factor for gambling disorder. Reward-paired cues may play an important role, as these stimuli can enhance risk preference in rats. Despite extensive research implicating the dorsal striatum in the compulsive aspects of addiction, the role of nigrostriatal dopaminergic activity in cue-induced risk preference remains unclear, particularly in females. Accordingly, we examined the effects of manipulating the dopaminergic nigrostriatal pathway on cue-induced risky choice in female rats. TH:Cre rats were trained on the cued version of the rat Gambling Task. This task was designed such that maximal reward is attained by avoiding the high-risk, high-reward options and instead favouring the options associated with lower per-trial gains, as they feature less frequent and shorter time-out penalties. Adding reward-paired audiovisual cues to the task leads to greater risky choice on average. To assess the role of the nigrostriatal pathway, a viral vector carrying either Cre-dependent inhibitory or excitatory DREADD was infused into the substantia nigra. Rats then received clozapine-N-oxide either during task acquisition or after a stable performance baseline was reached. Inhibition of this pathway accelerated the development of risk preference in early sessions and increased risky choice during performance, but long-term inhibition actually improved decision making. Activation of this pathway had minimal effects. These results provide evidence for the involvement of the dopaminergic nigrostriatal pathway in cue-induced risk preference in females, therefore shedding light on its role in cost/benefit decision-making deficits and expanding our knowledge of the female dopaminergic system.

Multiple timescales of temporal context in risky choice: Behavioral identification and relationships to physiological arousal.

Context-dependence is fundamental to risky monetary decision-making. A growing body of evidence suggests that temporal context, or recent events, alters risk-taking at a minimum of three timescales: immediate (e.g. trial-by-trial), neighborhood (e.g. a group of consecutive trials), and global (e.g. task-level). To examine context effects, we created a novel monetary choice set with intentional temporal structure in which option values shifted between multiple levels of value magnitude ("contexts") several times over the course of the task. This structure allowed us to examine whether effects of each timescale were simultaneously present in risky choice behavior and the potential mechanistic role of arousal, an established correlate of risk-taking, in context-dependency. We found that risk-taking was sensitive to immediate, neighborhood, and global timescales: risk-taking decreased following large (vs. small) outcome amounts, increased following large positive (but not negative) shifts in context, and increased when cumulative earnings exceeded expectations. We quantified arousal with skin conductance responses, which were related to the global timescale, increasing with cumulative earnings, suggesting that physiological arousal captures a task-level assessment of performance. Our results both replicate and extend prior research by demonstrating that risky decision-making is consistently dynamic at multiple timescales and that the role of arousal in risk-taking extends to some, but not all timescales of context-dependence.

Predictions about reward outcomes in rhesus monkeys.

Human infants and nonhuman animals respond to surprising events by looking longer at unexpected than expected situations. These looking responses provide core cognitive evidence that nonverbal minds make predictions about possible outcomes and detect when these predictions fail to match reality. We propose that this phenomenon has crucial parallels with the processes of reward prediction error, indexing the difference between expected and actual reward outcomes. Most work on reward prediction errors to date involves neurobiological techniques that cannot be implemented in many relevant populations, so we developed a novel behavioral task to assess monkeys' predictions about reward outcomes using looking time responses. In Study 1, we tested how semi-free-ranging monkeys (n = 210) responded to positive error (more rewards than expected), negative error (less rewards than expected), and a number control. We found that monkeys looked longer at a given reward when it was unexpectedly large or small, compared to when the same quantity was expected. In Study 2, we compared responses in the positive error condition in monkeys ranging from infancy to old age (n = 363), to assess lifespan changes in sensitivity to reward predictions. We found that adolescent monkeys showed heightened responses to unexpected rewards, similar to patterns seen in humans, but showed no changes during aging. These results suggest that monkeys' looking responses can be used to track their predictions about rewards, and that monkeys share some developmental signatures of reward sensitivity with humans, providing a new approach to access cognitive processes underlying reward-based decision making. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A critical review of risk-sensitive foraging.

Foraging is risk sensitive if choices depend on the variability of returns from the options as well as their mean return. Risk-sensitive foraging is important in behavioural ecology, psychology and neurophysiology. It has been explained both in terms of mechanisms and in terms of evolutionary advantage. We provide a critical review, evaluating both mechanistic and evolutionary accounts. Some derivations of risk sensitivity from mechanistic models based on psychophysics are not convincing because they depend on an inappropriate use of Jensen's inequality. Attempts have been made to link risk sensitivity to the ecology of a species, but again these are not convincing. The field of risk-sensitive foraging has provided a focus for theoretical and empirical work and has yielded important insights, but we lack a simple and empirically defendable general account of it in either mechanistic or evolutionary terms. However, empirical analysis of choice sequences under theoretically motivated experimental designs and environmental settings appears a promising avenue for mapping the scope and relative merits of existing theories. Simply put, the devil is in the sequence.

Option similarity modulates the link between choice and memory.

Choices made in everyday life are highly variable. Sometimes, you may find yourself choosing between two similar options (e.g., breakfast foods to eat) and other times between two dissimilar options (e.g., what to buy with a gift certificate). The goal of the present study was to understand how the similarity of choice options affects our ability to remember what we choose and what we did not choose. We hypothesized that choosing between similar as compared to dissimilar options would evoke a comparison-based strategy (evaluating options with respect to one another), fostering a relational form of encoding and leading to better memory for both the chosen and unchosen options. In Experiment 1, participants reported their strategy when choosing between pairs of similar or dissimilar options, revealing that participants were more likely to use a comparison-based strategy when faced with similar options. In Experiment 2, we tested memory after participants made choices between similar or dissimilar options, finding improved memory for both chosen and unchosen options from the similar compared to dissimilar choice trials. In Experiment 3, we examined strategy use when choosing between pairs of similar or dissimilar options and memory for these options. Replicating and extending the results of the first two experiments, we found that participants were more likely to use a comparison-based strategy when choosing between similar than dissimilar options, and that the positive effect of similarity on memory was stronger for unchosen than chosen options when controlling for strategy use. We interpret our results as evidence that option similarity impacts the mnemonic processes used during choice, altering what we encode and ultimately remember about our choices.

Adaptive decision-making depends on pupil-linked arousal in rats performing tactile discrimination tasks.

Perceptual decision-making is a dynamic cognitive process and is shaped by many factors, including behavioral state, reward contingency, and sensory environment. To understand the extent to which adaptive behavior in decision-making is dependent on pupil-linked arousal, we trained head-fixed rats to perform perceptual decision-making tasks and systematically manipulated the probability of Go and No-go stimuli while simultaneously measuring their pupil size in the tasks. Our data demonstrated that the animals adaptively modified their behavior in response to the changes in the sensory environment. The response probability to both Go and No-go stimuli decreased as the probability of the Go stimulus being presented decreased. Analyses within the signal detection theory framework showed that while the animals' perceptual sensitivity was invariant, their decision criterion increased as the probability of the Go stimulus decreased. Simulation results indicated that the adaptive increase in the decision criterion will increase possible water rewards during the task. Moreover, the adaptive decision-making is dependent on pupil-linked arousal as the increase in the decision criterion was the largest during low pupil-linked arousal periods. Taken together, our results demonstrated that the rats were able to adjust their decision-making to maximize rewards in the tasks, and that adaptive behavior in perceptual decision-making is dependent on pupil-linked arousal.NEW & NOTEWORTHY Perceptual decision-making is a dynamic cognitive process and is shaped by many factors. However, the extent to which changes in sensory environment result in adaptive decision-making remains poorly understood. Our data provided new experimental evidence demonstrating that the rats were able to adaptively modify their decision criterion to maximize water reward in response to changes in the statistics of the sensory environment. Furthermore, the adaptive decision-making is dependent on pupil-linked arousal.

Behavioral read-out from population value signals in primate orbitofrontal cortex.

The primate orbitofrontal cortex (OFC) has long been recognized for its role in value-based decisions; however, the exact mechanism linking value representations in the OFC to decision outcomes has remained elusive. Here, to address this question, we show, in non-human primates, that trial-wise variability in choices can be explained by variability in value signals decoded from many simultaneously recorded OFC neurons. Mechanistically, this relationship is consistent with the projection of activity within a low-dimensional value-encoding subspace onto a potentially higher-dimensional, behaviorally potent output subspace. Identifying this neural-behavioral link answers longstanding questions about the role of the OFC in economic decision-making and suggests population-level read-out mechanisms for the OFC similar to those recently identified in sensory and motor cortex.

Neural landscape diffusion resolves conflicts between needs across time.

Animals perform flexible goal-directed behaviours to satisfy their basic physiological needs1-12. However, little is known about how unitary behaviours are chosen under conflicting needs. Here we reveal principles by which the brain resolves such conflicts between needs across time. We developed an experimental paradigm in which a hungry and thirsty mouse is given free choices between equidistant food and water. We found that mice collect need-appropriate rewards by structuring their choices into persistent bouts with stochastic transitions. High-density electrophysiological recordings during this behaviour revealed distributed single neuron and neuronal population correlates of a persistent internal goal state guiding future choices of the mouse. We captured these phenomena with a mathematical model describing a global need state that noisily diffuses across a shifting energy landscape. Model simulations successfully predicted behavioural and neural data, including population neural dynamics before choice transitions and in response to optogenetic thirst stimulation. These results provide a general framework for resolving conflicts between needs across time, rooted in the emergent properties of need-dependent state persistence and noise-driven shifts between behavioural goals.

Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex.

Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.

Further observations on the reinforcing value of sucrose solutions: Interaction between quantity and concentration.

According to the Multiplicative Hyperbolic Model of reinforcer value (MHM), the value of a reinforcer is an increasing hyperbolic function of its size. In addition it is generally accepted that in the case of soluble reinforcers such as sucrose, value is an increasing function of molar concentration. The present experiment examined the interaction between size and concentration on the effectiveness of sucrose reinforcers in an operant choice paradigm. Rats were trained under an adjusting-magnitude schedule in which a response on lever B delivered a fixed volume of a sucrose solution, while a response on lever A delivered a 0.8-M sucrose solution, the volume of which, qA, was adjusted according to the rats' choices. When B was preferred in a given block of trials, qA was increased in the following block; when A was preferred, qA was reduced in the following block. The concentration and volume of B (cB, qB) were varied across six phases of the experiment and the corresponding indifference magnitudes of A (qA(50)) were measured. Log qA(50) increased and qB/qA(50) declined as approximately linear functions of log cB, consistent with the supposition that reinforcer value was determined by multiplicative combination of hyperbolic expressions representing concentration and volume.

The road not taken: Common and distinct neural correlates of regret and relief.

Humans anticipate and evaluate both obtained and counterfactual outcomes - outcomes that could have been had an alternate decision been taken - and experience associated emotions of regret and relief. Although many functional magnetic resonance imaging (fMRI) studies have examined the neural correlates of these emotions, there is substantial heterogeneity in their results. We conducted coordinate-based ALE and network-based ANM meta-analysis of fMRI studies of experienced regret and relief to examine commonalities and differences in their neural correlates. Regionally, we observed that the experience of both regret and relief was associated with greater activation in the right ventral striatum (VS), which is implicated in tracking reward prediction error. At the network level, regret and relief shared the reward-sensitive mesocorticolimbic network with preferential activation of the medial orbitofrontal cortex (mOFC) for regret processing and medial cingulate cortex (MCC) for relief processing. Our research identified shared and separable brain systems subserving regret and relief experience, which may inform the treatment of regret-related mood disorders.

Evidence accumulation is not essential for generating intertemporal preference: A comparison of dynamic cognitive models of matching tasks.

Intertemporal preference has been investigated mainly with a choice paradigm. However, a matching paradigm might be more informative for a proper inference about intertemporal preference and a deep understanding of the underlying cognitive mechanisms. This research involved two empirical studies using the matching paradigm and compared various corresponding dynamic models. These models were developed under either the framework of decision field theory, an exemplar theory assuming evidence accumulation, or a non-evidence-accumulation framework built upon the well-established notions of random utility and discrimination threshold (i.e., the RUDT framework). Most of these models were alternative-based whereas the others were attribute-based ones. Participants in Study 1 were required to fill in the amount of an immediate stimulus to make it as attractive as a delayed stimulus, whereas those in Study 2 needed to accomplish a more general matching task in which either the payoff amount or delay length of one stimulus was missing. Consistent behavioral regularities regarding both matching values and response times were revealed in these studies. The results of model comparison favored in general the RUDT framework as well as an attribute-based perspective on intertemporal preference. In addition, the predicted matching values and response times of the best RUDT model were also highly correlated with the observed data and replicated most observed behavioral regularities. Together, this research and previous modeling work on intertemporal choice suggest that evidence accumulation is not essential for generating intertemporal preference. Future research should examine the validity of the new framework in other preferential decisions for a more stringent test of the framework.

Working memory and attention in choice.

We study the role of attention and working memory in choices where options are presented sequentially rather than simultaneously. We build a model where a costly attention effort is chosen, which can vary over time. Evidence is accumulated proportionally to this effort and the utility of the reward. Crucially, the evidence accumulated decays over time. Optimal attention allocation maximizes expected utility from final choice; the optimal solution takes the decay into account, so attention is preferentially devoted to later times; but convexity of the flow attention cost prevents it from being concentrated near the end. We test this model with a choice experiment where participants observe sequentially two options. In our data the option presented first is, everything else being equal, significantly less likely to be chosen. This recency effect has a natural explanation with appropriate parameter values in our model of leaky evidence accumulation, where the decline is stronger for the option observed first. Analysis of choice, response time and brain imaging data provide support for the model. Working memory plays an essential role. The recency bias is stronger for participants with weaker performance in working memory tasks. Also activity in parietal areas, coding the stored value in working, declines over time as predicted.

Abstract perceptual choice signals during action-linked decisions in the human brain.

Humans can make abstract choices independent of motor actions. However, in laboratory tasks, choices are typically reported with an associated action. Consequentially, knowledge about the neural representation of abstract choices is sparse, and choices are often thought to evolve as motor intentions. Here, we show that in the human brain, perceptual choices are represented in an abstract, motor-independent manner, even when they are directly linked to an action. We measured MEG signals while participants made choices with known or unknown motor response mapping. Using multivariate decoding, we quantified stimulus, perceptual choice, and motor response information with distinct cortical distributions. Choice representations were invariant to whether the response mapping was known during stimulus presentation, and they occupied a distinct representational space from motor signals. As expected from an internal decision variable, they were informed by the stimuli, and their strength predicted decision confidence and accuracy. Our results demonstrate abstract neural choice signals that generalize to action-linked decisions, suggesting a general role of an abstract choice stage in human decision-making.

Reliable population code for subjective economic value from heterogeneous neuronal signals in primate orbitofrontal cortex.

Behavior-related neuronal signals often vary between neurons, which might reflect the unreliability of individual neurons or a truly heterogeneous code. This notion may also apply to economic ("value-based") choices and the underlying reward signals. Reward value is subjective and can be described by a nonlinearly weighted magnitude (utility) and probability. Defining subjective values relies on the continuity axiom, whose testing involves structured variations of a wide range of reward magnitudes and probabilities. Axiom compliance demonstrates understanding of the stimuli and the meaningful character of choices. Using these tests, we investigated the encoding of subjective economic value by neurons in a key economic-decision structure of the monkey brain, the orbitofrontal cortex (OFC). We found that individual neurons carry heterogeneous neuronal value signals that largely fail to match the animal's choices. However, neuronal population signals matched the animal's choices well, suggesting accurate subjective economic value encoding by a heterogeneous population of unreliable neurons.